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Abstract  ~  This  paper  describes  the  measurement  of  various  head 
images  and  the  presentation  of  integrated  images  on  an 
immersive  projection  system  (IPS).  We  handled  optical  and 
magnetic  method  to  obtain  the  information  about  face,  head, 
brain  and  blood  vessel.  The  system  enables  to  observe  the  inside 
of  a  head  interactively  using  a  joystick,  which  gives  the 
impression  to  travel  in  a  human  head  freely.  It  is  thought  that  the 
integrated  images  are  available  not  only  for  diagnosis  but  also 
treatment  of  a  brain  in  the  future  robot  surgical  operation  with  a 
manipulator  or  a  micro-machining  device. 
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I.  Introduction 

The  progress  of  medical  imaging  provides  us  many  kinds  of 
human  information,  morphology,  function,  metabolism  and  so 
on,  with  various  medical  equipments  [1].  For  example, 
anatomy  of  a  human  head  can  be  revealed  with  magnetic 
resonance  imaging  (MRI),  structure  of  brain  blood  vessels 
with  magnetic  resonance  angiography  (MRA)  and  brain 
metabolism  with  positron  emission  tomography  (PET).  Each 
medical  image  had  been  presented  respectively,  but  high 
computer  performance  enables  to  integrate  various  medical 
images  with  matching  their  coordinate  systems  [2]. 

Ten  years  ago  or  more,  such  kinds  of  information  were 
presented  as  a  two-dimensional  (2D)  image  and  a  medical 
doctor  was  forced  to  guess  the  three-dimensional  (3D) 
structure  of  an  organ,  a  tissue  and  so  on.  The  recent 
development  of  computer  graphics  (CG)[3]  resulted  in 
reconstructing  a  3D  image  from  many  2D  images.  As  a  3D 
image  can  be  observed  from  any  direction,  a  3D  medical 
image  is  effective  to  educate  a  medical  student  and  very  useful 
for  diagnose  and  the  plan  before  a  surgery  [4]. 

Virtual  reality  (VR)[5]  and  CG  technique  realize  the 
fantastic  world  that  we  have  never  experimented.  However, 
the  presentation  of  a  3D  image  on  a  cathode  ray  display  (CRT) 
does  not  give  immersive  feeling  to  us.  Nowadays,  the 
immersive  projection  system  (IPS)  has  been  developed  using 
some  large  screens  [6],  which  presents  us  the  feeling  as  if  we 
were  in  a  human  body  [7].  Here,  we  obtained  various  kinds  of 
information  about  a  head,  integrated  them  and  projected  on  a 
CRT  or  an  IPS.  This  document  reports  the  measurement 
method,  the  reconstructing  process  and  the  presentation  of  the 
results  on  a  CRT  or  an  IPS. 


II.  Method 

A.  Face  Shape 

We  measured  head  shape  using  a  range  finder  (Cyberware: 
Head  &  Face  3D  Scanner)  [8].  On  measurement,  a  subject  sat 
on  the  chair  that  rotated  at  the  constant  angular  velocity  and 
the  distance  between  face  surface  and  rotation  axis  was 
measured.  Simultaneously,  a  texture  image  of  a  face  was 
captured.  The  range  finder  acquires  the  face  shape  and  texture 
512  times  in  a  rotation  and  in  each  acquisition  the 
measurements  were  performed  at  450  points  on  a  line  in  the 
pitch  of  0.7mm.  Fig.l  shows  range  data,  the  texture  image  of  a 
face  and  the  patched  texture  image  on  range  data. 


B.  Brain  Morphology 


A  head  shape  and  a  brain  surface  were  obtained  from  MRI 
slices  measured  with  an  MRI  scanner  (Siemens:  1.5T  MR 
scanner)  [9].  As  imaging  sequence,  we  chose  gradient  echo 
method  combining  fast  imaging  method  (FLASH:  fast  low 
angle  shot)  with  the  flip  angle  of  8 deg,  time  of  repetition  Tr  of 
11.6msec  and  time  of  echo  Te  of  4.9msec  [10].  Table  I 
summaries  the  parameters  of  MRI  slices  and  Fig.2  shows  an 
example  image  of  MRI  slices. 


Table  i 


PARAMETERS  OF  MRI  SLICES 


Image  Size 

512  x  512pixel 

Image  Resolution 

0.5  x  0.5mm/pixel 

Slice  Thickness 

0.5mm 

Image  Depth 

12bits 

Slice  Number 

256 
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C.  Brain  Blood  Vessel 


Gray 

Matter 


Head  Surface 


Fig.2  MRI  image 

A  head  shape  image  was  reconstructed  with  extracting  the 
head  surface  edge.  While  a  brain  surface  image  was  done  with 
the  method  described  below.  Fig. 3  is  a  histogram  of  an  MRI 
image.  On  the  histogram,  a  peak  appears  in  relative  high  gray 
level  marked  by  ‘A’,  which  corresponds  to  gray  matter. 
Therefore,  the  threshold  value  was  set  at  the  minimum  value 
marked  by  ‘B’  [11].  After  binarization,  dilation,  erosion  and 
labeling  procedure  were  performed,  the  labeled  regions  with 
larger  area  were  selected  as  gray  matter  as  shown  Fig.4  and 
the  outline  of  gray  matter  was  extracted.  These  procedures 
were  applied  to  all  MRI  slices  and  a  3D  brain  surface  image 
was  reconstructed. 


Fig.3  Histogram  of  MRI  image 


Fig.4  Image  of  extracted  gray  matter 


A  3D  brain  blood  vessel  image  was  reconstructed  from 
MRA  slices  obtained  with  3D-TOF  (time  of  flight)  based  on 
gradient  echo  method  [10].  Table  II  summaries  the  parameters 
of  MRA  slices  and  Fig. 5  shows  an  example  image  of  MRA 
slices. 

Table  ii 


PARAMETERS  OF  MRA  SLICE 


Image  Size 

512  x  512pixel 

Image  Resolution 

0.5  x  0.5mm/pixel 

Slice  Thickness 

0.5mm 

Image  Depth 

12bits 

Slice  Number 

228 

Fig. 5  MRA  image 
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On  MRA  images,  blood  vessels  were  represented  as 
brighter  spots  shown  in  Fig. 5.  The  slice  or  the  distance  from 
the  brain  midline,  however,  varied  the  gray  level  of  blood 


vessels.  Therefore,  the  threshold  value  was  determined  for 
each  slice  [7]  and  labeling  procedure  followed. 

Then,  connectivity  was  examined  between  adjacent  slices  as 
shown  in  Fig. 6,  where  the  labeled  regions  were  supposed  to  be 
connected  each  other  whether  only  one  pixel  was  overlapped 
between  the  slices.  The  area  and  gravity  were  calculated  for 
the  labeled  regions  and  blood  vessels  were  painted  in  a 
hexagon  shape  as  shown  in  Fig. 7.  These  procedures  were 
applied  to  all  MRA  slices  and  a  3D  blood  vessel  image  was 
reconstructed. 

III.  Presentation 
A.  Presentation  on  a  CRT 

As  described  in  the  last  section,  we  reconstructed  several 
images  of  a  head.  Fig.  8  shows  these  images  presented  on  a 
CRT  sequentially.  The  matching  of  coordinate  systems 
between  face  and  head  shape  was  performed  manually.  No 
matching  was  carried  out  between  head  shape  and  brain 
surface  and  between  brain  surface  and  blood  vessel.  Fig.  8 
demonstrates  smooth  morphing  from  face  to  brain  vessel  and 
various  kinds  of  head  information  could  be  observed  from 
many  viewpoints. 


(a)  (b)  (c) 


(d)  (e)  (f)  (g) 

Fig.  8  Presentation  of  each  image  on  CRT  (a)  face  shape  (b)  face  and  head 
shape  (c)  head  shape  (d)  head  shape  and  brain  surface  (e)  brain  surface  (f) 
brain  surface  and  blood  vessel  (g)  blood  vessel 

B.  Presentation  on  an  IPS 

After  extracting  gray  matter,  256  slices  of  512  x  512pixel 
was  converted  to  512  x  512  x  512voxel  using  linear 
interpolation.  Three  surfaces  of  80  x  80  x  60 voxel  data  among 
the  original  voxel  data  were  drawn  on  three  screens 
transparently  so  that  blood  vessels  can  be  seen  through  MRI 
images. 


(a)  (b)  (c) 

Fig.9  Projection  of  MRI  images  (a)  512  x  512  x  512voxel  (b)  80  x  80  x 
60voxel  (c)  projection  on  three  screens 


A  head  shape,  a  brain  vessel  and  MRI  images  were 
projected  on  an  IPS.  Fig.  10  shows  the  IPS  that  has  four 
screens  with  the  size  of  3  x  2.3m.  On  this  presentation,  these 
images  were  projected  on  center,  left  and  right  screen  only. 
This  system  possesses  a  pointing  device,  a  joystick  as  seen  in 
Fig.9,  which  was  used  to  move  a  viewpoint  and  direction 
interactively. 


Fig.  10  Immersive  projection  system 


Fig.  11  shows  the  presentation  of  3D  head  shape  on  an  IPS 
just  before  going  into  a  head.  Fig.  12  shows  blood  vessels  and 
MRI  images.  Some  vessels  could  be  seen  through  MRI 
images. 


Fig.l  1  3D  image  of  head  shape  projected  on  an  IPS 


Fig.  12  3D  image  of  MRI  and  blood  vessel  projected  on  an 


Fig.  13  shows  3D  head  shape  and  brain  vessels  with  the  eye 
position  set  in  the  head.  These  images  do  not  involve  MRI 
images  and  another  side  of  the  head  surface  can  be  observed. 
Fig.  13  also  indicates  interactive  observation  of  the  3D  image 
using  a  joystick.  It  is  easy  to  change  viewpoint  and  viewing 
direction  because  the  change  was  performed  using  a  joystick 
that  is  one  of  the  most  comfortable  pointing  devices. 
Moreover,  this  system  provided  us  the  walk  through  in  a 
human  head  and  guided  us  to  go  into  and  out  of  a  head  freely. 


(b) 

Fig.  13  Interactive  observation  (a)  seeing  head  shape  and  blood  vessel  (b) 
moving  viewpoint  slightly  upward 

IV.  Conclusion 

We  obtained  various  kinds  of  head  information  with  the 
optical  and  magnetic  method  and  projected  the  reconstructed 
images  on  a  CRT  or  an  IPS.  The  IPS  has  the  function  of 
interactive  observation,  which  makes  the  spatial  relationship 
among  tissues  clear.  Furthermore,  this  system  gives  us  the 
immersive  environment  that  we  are  staying  in  a  human  body. 
Therefore,  the  system  is  thought  to  be  useful  for  a  catheter 
surgery  as  monitoring  system  [12].  In  the  future,  this  system 
will  be  the  essential  tool  for  robot  surgery  with  a  manipulator 
[13]  and  an  ongoing  micro-machining  device  [14]. 
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